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METHOD AND APPARATUS FOR P^EPROCESSin^ 
CONTENT IN AN INTERACTIVE INFORMATION DISTRIBUTION SYS iliM 

rROSS-REFFT^FNCF TO P FT ATRD APPT JCATION 
5 This invention claims benefit of U.S. Provisional Patent Applications Serial Nos. 

60/178,795, 60/178,809, 60/178, 810 and 60/178,857, all filed on Janilary 28, 2000, and 
such applications are herein incorporated by reference in their entireties. 

This invention is related to simultaneously filed U.S. Patent Applications Serial 

Nos. (Attorney Docket No. DIVA 253) and (Attorney 

10 Docket No. 256), filed on the same date as this application, and such applications are herein 
incorporated by reference in their entireties. 

RAr.KnROTTNF) OF THE TNVF.NTION 



15 1. Field of the Invention 

The invention relates to electronic storage and transmission of content. More 
particularly, the invention relates to a method and apparatus for preprocessing and 
postprocessing content in an interactive information distribution system. 

20 2. Description of the Background Art 

Information systems such as video on demand (VOD) systems are capable of 
streaming program content to a great number of users or subscribers. To provide a 
requested program content to a subscnber. the VOD system retrieves the requested program 
content from a video server, streams the content over a stream distribution network, and 

25 converts the content to an access network that is coupled to a particular neighborhood of 
subscriber terminals.. The user then v,ews the requested program content at the subscriber 
terminal. 

However, the different types of access networks have different limitations with 
respect to transmission latency, bandw.dth, and the like. To service a wide subscriber base, 
30 the VOD systems currently implement different solutions for each type of access network. 
For example, VOD systems that provide web-based video content must account for a public 
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and private w.de area networks that support content of a particular quality of service (QoS), 
typically medium latency, low bandwidth and poor quality video, e.g. high jitter. 
Additionally, VOD systems that provide cable-based video must account for cable 
networks that support low latency, high bandwidth and high quality video. 
5 One example of using different solutions involves the use of separate video servers 

for each type of access network. Such a solution only increases the cost of providing 
program content at the head end. Therefore, there .s a need in the art to provide a scalable 
VOD solution that is common for the different types of access networks. Additionally, 
there is a need to preprocess and postprocess content for such a VOD solution. 

10 

qTTMMARV rsv: THP INVENTION 
The invention provides a method and system for preprocessing content for a stream 
caching server in an interactive information distribution system. In one embodiment, the 
method initially retrieves of content in a subscriber terminal. The retrieved content is 
15 encapsulated in accordance to an Internet Protocol (IP) format used for streaming content to 
various access networks. The encapsulated content is then uploaded for storage in a stream 
caching server and for future streaming of content to different types of access networks. 

The present invention preprocesses content into a common format suitable for a 
stream caching server capable of transmitting content to different types of access networks. 
20 In one embodiment of the invention, a user executes an applet program to preprocess 
content stored in a computer terminal. Such a configuration enables a user to upload 
content over the Internet for storage in a stream caching server and subsequent streammg to 
other subscribers. The invention also postprocesses content into a format supported by a 
particular type of player and access network used to receive the content from the stream 
25 caching server. 

RRTRF DF^^r-RTPTTON OF THF. DRAWINGS 
The teachings of the present invention can be readily understood by considering the 
following detailed description in conjunction with the accompanying drawings, in which: 
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FIG. 1 depicts a high level block diagram of a first portion of an interactive 
information distribution system embodied in the present invention; 

FIG. 2A depicts one embodiment of an Internet Protocol (IP) packet used in the 
information distribution system of FIG. 1; 
5 FIG. 2B depicts one embodiment of a Realtime Transport Packet (RTP) contained 

in a payload section of the IP packet of FIG. 2A; 

FIG. 3 depicts a data structure useful in understanding an embodiment of the present 

invention; 

FIG. 4 depicts a flow diagram of a method for implementing the preprocessing of 
10 program content in accordance to one embodiment of the present invention; and 

FIG. 5 depicts a flow diagram of a method for and postprocessing content in 
accordance to another embodiment of the present invention. 

To facilitate understanding, identical reference numerals have been used, where 
possible, to designate identical elements that are common to the figures. 

15 

DFTATT ED PFSCRTPTION 
FIG. 1 depicts a high level block diagram of an interactive information distribution 
system 100. One application of the distribution system 100 is as a video of demand (VOD) 
system, as described in U.S. Patent Application No. 08/984,710, filed December 3, 1997 
20 and incorporated herein by reference. In such a VOD system 100, a user may request and 
receive a particular content selection, e.g., video, movie or programming content, from a 
service provider without any time restrictions associated with normal cable programming. 

The information distribution system 100 comprises a stream caching server 102, a 
stream distribution network 104, at least one access network and at least one subscriber 
25 terminal. The stream caching server 102 receives, stores and streams content in accordance 
to an Internet Protocol (IP). One example of such a stream caching server 102 is disclosed 

in simultaneously filed U.S. Application . attorney docket DIVA 253, entitled 

"Method and Apparatus for Streaming Content in an Interactive Information Distribution 
System, which is herein incorporated by reference. The content is configured within a 
30 payload portion of each IP packet received, stored and streamed by the stream caching 
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server 102. The use of this IP formatted content enables a single stream caching server 102 
to stream content via an integrated stream distribution network 104 to different types of 
access networks. As such, the system 100 is capable of streaming the same content to any 
cable service subscriber or any person using the Internet. 
5 In accordance to the present invention, the stream caching server 102 may receive 

content that is preprocessed at, for example, a remotely located subscriber terminal. One 
such subscriber terminal is a computer terminal 1 16 comprising a processor 150, a storage 
medium 152, a random access memory (RAM) 154 and support circuits 156. The RAM 
154 stores an applet 154 that is downloaded from, for example, a HTTP server 148 coupled 
10 to an access network, for example., a private local area network (LAN) or wide area 
network (WAN) 106. The processor 150 executes the applet 154 to initiate the 
preprocessing in the present invention. The storage medium 152 stores the content to be 
preprocessed. The support circuits 156 provide an interface for receiving the applet 154 
from the http server 148, receiving content from the streaming cache server 102, or 
15 uploading preprocessed content to the http server 148. 

Ipossible configurations of the applet 154 include a JAVA Applet Plug In for an 
(browser, or a software program written in a particular programming language, e.g., 
- Ince the processor 150 executes the applet 154, the content is retrieved from the 
Ledium 152. The content may include standard multimedia files in a variety of 
le.g., AVI (Audio Video Interleaved), Moving JPEG, MPEG-1, MPEG-2, MPEG- 
- 4, MP3, Quicktime, and the like. If a need exists to convert the content into a particular 
format, the retrieved content may be transcoded into a format supported by a viewer or 
subscriber terminal that eventually receives the downstream content. The transcoding of 
content changes the format and rate of the retrieved content. One example of such 
25 transcoding is the conversion of MPEG-2 content into MPEG-4 content that can be played 
on a graphic processor in set top terminals or personal computer (PC) terminals. Other 
types of content may be transcoded into MPEG-2 content playable on conventional set top 
terminals. Some transcoding requires decoding to baseband followed by encoding 
according to the desired format. Some transcoding may be performed without baseband 
30 decoding. 
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The content, whether transcoded or not, is then encapsulated into a format that is 
optimal for the stream caching server 102. In one embodiment, the content, as MPEG-2 
packets, is encapsulated in a payload of each Realtime Transfer Protocol (RTF) packet 
contained in the payload of an IP packet. The format of such encapsulated content is 
5 shown in FIGS. 2A-2B. However, the use of the RTF packet also supports non real time 
applications. 

FIG. 2A depicts one embodiment of an Internet Protocol (IF) packet 200 used in the 
present invention. The IF packet comprises an IP header 210 and an IP payload 320. The 
IP payload comprises a UDP (User Datagram Protocol) packet 221 comprising a UDP 

10 header 222 and a UDP payload 224. A RTF packet 230, a stream integrity check 226 and a 
cyclic redundancy check (CRC) 228 is illustratively contained in the UDP payload 224. In 
one embodiment of the IP packet 200, the IF header 210 is 20 bytes, the UDP header 222 is 
8 bytes, the stream integrity check field 226 is 4 bytes and the CRC field 228 is 4 bytes. 
FIG. 2B depicts one embodiment of a Realtime Transport Packet (RTF) 230 

15 contained in a payload section 220 of the IP packet 300 of FIG.2A. The RTF packet 230 
comprises a RTF header 240 and a RTF payload 250. Five MFEG-2 packets 352, 354, 356, 
358 and 360 are illustratively contained in each RTF payload 250. The number of MFEG-2 
packets in the RFT payload 250 corresponds to the buffer space in the Fibre Channel 
controller in the packet processor 144. 

20 For the embodiment shown in FIGS. 3A-3B, the transcoding may also include 

compression of the IP header 310, the UDP header 322 and the RTF header 340. The 
compression of these headers 310, 322 and 340 optimizes the storage of content on the 
storage medium 146 of the stream caching server 102. Additionally, the transcoding may 
include encryption of the content. 

25 Compression of the IP header 310 may include compression of non-address fields, 

and deletion of source and destination IP addresses. The IP source address is subsequently 
decompressed (at the packet processor 144) as the IP address at the output interface of the 
stream caching server 102. The IP destination address is assigned prior to streaming and is 
based upon the data link converter 1 12, 1 18 or 126 or edge device for the target access 

30 network 106, 108 or 1 10. 
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The compression of the UDP header 322 may delete source and destination port 
numbers in the storage medium 146. New values are then assigned to the source and 
destination port numbers prior to streaming of content over the stream distribution network 
104. The source port number is assigned a unique stream number with IP addresses. The 
5 destination port number is assigned a unique target stream number within the data link 
converter 1 12, 1 18 or 126 or the target edge device. 

Once encapsulated into the IP format, the content is then uploaded to the http server 
148 via the modem 1 14 and the LAN/WAN 106. The HTTP server 148 provides a user 
interface, e.g., a HTML (HyperText Markup Language) page, for the user to upload the 
10 content to the stream caching server 102. The http server 148 also transmits the transcoded 
content upstream to the data link converter 1 12 via the LANAVAN 106. The data link 
converter 1 12 modulates the encapsulated content for upstream transmission over the 
distribution network 104 for storage in the stream caching server 102. 

The preprocessing of the present invention is not limited to a computer terminal 1 16 
15 uploading content over a LANAVAN 106. For example, the encapsulation may also occur 
in the computer terminal 1 16 prior to uploading to content to the http server 148. 
Additionally, the preprocessing may be initiated from a computer terminal 122 or digital 
video recorder 124 over a carrier network 108, e.g., a T-1 or T-3 line. As such, a user of 
any computer terminal 1 16 and 122 author multimedia content over a network, e.g., 
20 Internet, and store the content in a virtual video shelf at the stream caching server 102 for 
playback by other users. The preprocessing is also applicable to content from a content 
provider such as a movie manufacturer. 

The preprocessing may also include the creation of metadata once the processor 150 
executes the applet 154. The metadata generally contains a variety of information about the 
25 content to be stored on the stream caching server 102 and streamed to a viewer or 

subscriber terminal. One embodiment of the metadata is in the form of a data structure that 
is prepended prior to a file associated with the content. 

The metadata may comprise many different types of information used by the stream 
caching server 102 in steaming the content to a viewer device or subscriber terminal. One 
30 type of metadata that identify the content include title, author, screenwriter, actors, length 
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of play, timmg information, play rate, e.g., constant b.t rate (CBR) or variable bit rate 
(VBR) genre of content, size of content, and the like. Other forms of metadata indicate the 
type of content and the type of player used to play the content. Illustrative types of content 
include AVI, MPEG-1, MPEG-2, MPEG-4, MP3, Quicktime, and Moving JPEG. The 
5 metadata may also include MPEG7 structure including scene descriptions and indices. 
Exemplary types of player devices include MPEG-1 player, MPEG-2 player, MPEG-4 
player, Microsoft Media Player, Real Video / Real Audio Player, and QuickTime Player. 

The metadata may include pricing information and restrictions to view the content 
from the server 102. A user or service provider may preset the price and access restrictions 
10 that are required to view the preprocessed content from the server 102. Pncing information 
include a price to view the content, applicable discounts associated with viewing the 
content, and applicable package deals when ordering the preprocessed content with other 
content selections. Access restrictions include rating of content, viewing window 
information and sales window information. The viewing window is a graphical interface 
15 that requires a subscriber to enter a correct password to receive the content from the server 
102. The sales window is a graphical interface that requires a subscnber to pay for viewing 

a particular content selection. 

The metadata comprises indexing at IP packet boundaries, e.g.. Group of Pictures 
(GOP) boundaries and frame boundaries. This indexing enables the stream caching server 
20 102 in responding to an interactive VCR like commands, e.g., fast forward (FF), rewind 
(REW) pause, stop, bookmark, and return to place. Specifically, the indexing information 
supports random frame access at a content stream granularity, separate FF and REW tracks, 
random frame access of FF and REW tracks, pause/play, bookmarks for each active 
subscriber, and DVD scene selection. Additionally, the metadata may also include markers 
25 for changes in variable bit rate (VBR) and statistical multiplexing. 

The indexing enables the use of MPEG-7 based descriptors for indexed access of 
content. The indexing of content can be on a per frame basis or a per GOP basis. MPEG-7 
based descriptors include a length of descriptor, i.e., from a start frame to an end frame, and 
a schema for a database of indices. The database is a hierarchical based database that 
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allows for hierarchical scene descript.on. For example, a root ,ndex may represent a scene 
of Pans, whUe a branch index may represent a scene of the Eiffel Tower. 

The indexing is also used for stream creation. In one embodiment, the stream .s 
created in realtime from a single MPEG-2 or MPEG-4 content stream, e.g., a start GOP to 
5 an end GOP, or a start frame to an end frame. In a second embodiment, the stream is 
created in non real time and the modified stream file is stored. In a third embodiment, the 
stream is transcoded or re-encoded such that a reference frame or I-frame is forced to be the 
frame with information desired in an index, even if the frame arrived in the packet 
processor 144 as a predictive frame, e.g., B-frame or P-frame. Stream indexing will also be 

10 discussed below with respect to FIG. 3. 

Once the preprocessed content ,s uploaded, the strean, caching server 102 may store 
and stream the prcprocessed content, or alternatively stream the content in real time^ The 
streaming of content encapsulated in IP format enables the stream cach.ng server 02 to 
stream content to subscribers via different types of access networks ,06, 108 and 1 10^ As 
,5 such only one stream cach.ng server 102 and one distrrbu.ton network 104 ,s required to 
provide scalable strcammg. Namely, one stream cach.ng server 102 may stream content to 
the LAN/WAN 106, the earner network 108, the cable network 1 10 and any other access 
network that supports IP. Th,s greatly teduces the hardware cos. at the head end 138, as the 
prior art requires a streaming caching server 102 and a distribut.on network 104 for each 
20 type of access network 106, 108 and 1 10. 

For example, the server 102 may stream content to a private Local Area Network 
(LAN) or Wide Area Network (WAN) 106. Specifically, the system 100 may stream 
content through the stream distribution network 104 to a digital link converter 1 12, the 
LAN or WAN 106, a modem 1 14 and a display device coupled to a computer terminal 1 16. 
.5 If a user decides to request content from the server 102 or uplink content, e.g., a home 

movie to the server 102, that request or content would travel upstream in a path reverse to 
that of the downstreamed content. The digital link converter 1 12 modulates the program 
content for transmission via the private LAN/WAN 106. Additionally, the data link 
converter 1 12 extracts a MPEG formatted program content from the RTP formatted stream 
30 from the stream distribution network 104 and transcodes the program content into a format 
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,ha. ,s supponed by ,he LAN/WAN 106. One example of the 6a.a link convener 1 12 . a 
B vI pIL, UnMDDL Sm .Ka. ^rfo^. ,uadra>u. amplUude n,odu,aUon (QAM, on 

downs„eam p.o.ran, con.en,. The LAN o,- WAN 106 Is a private ne.wo* prov.ded by 
a pnva„ pariy or an In.e™. Serv.ce Provider (ISP). The ™oden, U4 demodulates v,deo 
5 content for viewing on the computer terminal 116. , ,w,04toa 

The server 102 may also s.,.am content via the stream d,stnbu„on network 104 to 
data hnk converter 1 18. a cat^er network 108, a digital subscnber line -ess mu lupiexer 
(DSLAM) 1 19. an x-DSL modem 120 to either a computer terminal 122 or a d,g,ta. v.deo 
larder 124. A request for content or upload of con.en. wouid travel m the reverse 

,0 pa.h .aken by the downst.am content. The carrier network 108 may tnciude a T or T-3 
ransmission hnk. The data Hnk convener ,18 mult.plexes .he "'^^J 
transmisston via ,he ca^cr network 108. Additionally, the data link convener 1 18 may 
extract MPEG packets from the IP forma..ed s.eam from .he s.ream 

104 The DSLAM 1 19 demul.,piexes the downstream content to a pan.cular xDSL mod m 
13 120 The XDSL modem 120 demodulates the content for vtewing on a computer term.nal 

emprise a ADSL (asynchronous dig.tal subscriber hne, modem, a VDSL (very h,gh data 
ra.e digiial subscriber Une). and the like. _ 
The server 100 may also s.,^am content (or program content selection) v,a the 
,0 stream dtstHbution network 104 to a da.a link convener 126. the cable -"orkJ^^O to 
either a set top ten^inal 128 or a cabie modem that ,s coupled to a computer «nnm 130 
a DVR 132. The data hnk convener 126 operates in a s.milar manner to the d,g« hnk 
convener 112 except to fonnat the con.en. for .ransmiss.on in the cable network 1 10. 
content is transmitted from the cable network 110 to a set top terminal 128 or a cable 
,5 modem 130 that demodulates the program content for viewing on a computer termmal 132 
' oradisplay devcecoupiedtotheDVR ,34. A request from a cable subscriber or user ,s 
processed v,a the cable network 1 10, the OOB (out of band, n,uter 136 and the modulator 
126 that modulates the request back to the stream distribution network 104. 

Although the system ,00 ,s illustratively shown ,0 stream pn.gram content to the 
,0 LAN/WAN 106, the PSTN 108 and the cable network 1 10, the system 100 may also stream 
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con,enC .o o.her .ypes of access ne.worKs. Fo. e.a.p.e. .Ke sys». .CO n.a also s„ea. 
program con.enc .o sateUUe and te^s.ria, ne.worRs. AddUionaUy, each system 100 
ZU s,rea.s content over .any .ore access networks and su.sc*er ,e.nn,na,s ,Han .He 

example shown in FIG. 1- ^ • ^ 

The stream caching server 102 is located a. the local head end 138 w,.h an 
infrastructure system manager 140, a switch 142 and a pacKet processor 144. The stream 
caching server 102 comprises a storage medium 146 to store the content preprocess^ n 
accordance to the present invention. One configurat.on of the storage —1 6 
.dundan. se, of disk anays, e.g., Redundant Array of Ine.pens.ve D.sks (R AID, 

The infrastructure system manager 140 coordinates a (user) request from the 
suhscnher terminal hy passms the ret,uest to the stream cachtng server 102 and estahhsh.ng 
a sesston between the subscnber terminal and the stream caching server 102. An 
exemplary infrastructure system manager 140 is the DIVA System Manager (DSM). 
^ ,. . Attorney docket DIVA 256, entitled 

disclosed in U.S. Apphcatton Attorney . ^ .„„„ 

,5 "Method and Apparatus for Managing an Integrated Inforrrration Distnbut.on S stem . 
„h,ch .s fully incorporated by reference ,n ,ts enttrety. The sw.tch 142 rou„s the us„ 
.equest from the stream distrrbution network .04 to the system manager ; 
th! switch 142 routes the retrieved content from the stream caching server .02 ,0 the packet 

20 '""Thetorage medium .48 stores the preprocessed content in an IP format. The 

content ,s configured as a plurality of MPEO. e.g., MPEO-2 or MPBC-4 - — 
in a payload of a RTP packet w.th.n an IP packet. For e.a,.ple, the payload of eac RTP 
packet may conta.n five MPE0.2 packets. The st,.ct„re of the IP packet ,s shown to HC. 
3B The RTP format (RFC 1889) minim.zes the latency in streaming content from the 
.5 server by supporting the streaming of content in real time. Add.tionaMy, the content in the 
Tp Ckelcan be configured to have a minimal tonality of Service (QoS). e.g.. d«a latency. 

The packet processor 144 postprocesses the content into a format supposed by 
particular type of player and access network 106, 108 and 1 10 used to receive the coit.en, 
Che stLam caching server 102. Such a player is e.ther a software module down oaded 
30 from a HTTP server 1 16 to a computer terminal .22. a hardware module coupled 
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subscriber .er..„a,, or a card .nserred in.o a subscriber .ernn.na.. Exemplary players 
inCude a MPEG-1 p.ayer, a MPEG.2 player, a MPEG-4 player, a Mlcrosof. Med.a Player, 
a Real V.deo/Real Audio P.ayer. a Qu.c.T.me Player, a W.reless Dev,ce V.deo or Aud.o 

Plaver and the like. . ■ u 

5 The packe, processor 144 iranscodes ,be content is performed witbou. disturbing tbe 

IP forma.. For example, the packet processor 144 separates the content, e.g., MPEG-2 

packets, and header tnfot^ation .n the ,P packet, transcodes the content packets into a 

desired format supported by the access network and downstrean, player, and combmes he 

^scoded packets with the header tnformation to recreate the ,P packet. Such transcodtng 

to ,s performed a, an elementary packet level for transm.tting at the transport packet level. 

Additional functions perfon^ed by the packet processor .44 include jitter cor^ctton 

creating of a PES (packet elementary stream,, stream splicng, and stafstical muiup.ex,„g. 

More specifically, the transcoding includes the conversion content ,n the RTP 

15 content is st,l. encapsulated in the IP packet stream. Such transcod.ng may change the 

format and rate of the content. For example, the transcoding may include the convers^n of 
MPEG-2 fo^aued content ,nto MPEG-1, MPEG-4, AVI, Moving IPEG, MP3, Qutckume, 
Wireless Applications Protocol content, and the like. The transcoding is performed ,„ 
accordance to an extended Real Time Streaming Protocol (RTSP - RFC 2326, such that 

,0 stream man.pula.tons confomt to Internet standards and are applicable to any access 

networks that support IP. ' 

Additionally, the exact manner of the transcoding depends on the available 

handwidth in the access network used to receive the content at the player. For example, the 
packet processor 144 may perform sta.istica, multiplexing to dynamically allocate dte 
25 amount of available bandwidth for streaming content to a particular viewer. To perform 
such statistical multiplexing, the packet processor 144 may stream content at either a 
constant bit rates or variable bit rates. 

The transcoding is also adjustable to bandwidth degradations. To process lossy 
video, the transcoding may include lossy filtering within frames of content, dropping 
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second, and delivenng sull frames .ha. con.a.n ,mpor,ant informa.ion. For non-lcssy 
c„n,press,on, .he transcoding may include dropping MPEG nu„ pacKCs, and .ranscod,ng or 
re-encoding content to an acceptable quality. 

The packet processor 144 may au.omat.cally perform such transcoding, or perform 
3 transcoding in accordance to user configured preferences. These preferences may tnc.ude 
choices for a particular player, e.g., formatt.ng. play rates, and type of converston or 
transcodtng. For example, ,f the player is embodted ,n software ,n a PC, then the con ent ,s 
transcoded into MPEG-2 forma.. However, ,f the player is a hand held device then the 
content is transcoded into ;PEO or MPEG-4 format. Additionally, the transcodtng may 
.0 dynamically performed based on a user preference profile. Such a profile ,s based e.ther on 
hLory or a default preference. For example, if the player ,s ,n the PC, .he packe. processor 
144 .ranscodes .he comen. into MPEG-2 at 4 Mbps and a constant b,t rate. 

The content m the payload of each RTP packet ,s sized to mihim.ze the latencies ,h 
streaming conten. from .he stream caching server .02 to the dis.ribut,on network 104. .The 
.3 read block for the packet processor 144 is s.zcd to the MPEG packets in the payload of 
each RTP packet. The number of MPEG packets in each RTP packet is cons.ramed by an 
available buffer space in a Fibre Channel controller .hat ,s used to read the content. 

The content streamed by the stteam caching server 102 is not limited to content 
previously sto«d ,n the storage medium 148. In one embodiment, the stream cachmg 
20 server .02 streams content from another remotely located server, t.e., a server located at a 
remote headend. Such a configuration is further described with respect to FIG. 2. 

The manager 140 provides sess.on managemcn. for streaming content m accordance 
to the RTP Control Protocol (RTCP). Such management ,s particularly .mportan, ,n the 
case of content streamed .o .he local s.ream cach.ng server 102 from .he remore server, f 
. any errors occurred during the stream.ng from .he remote server, ------ - 

. when the cached or s.ored con.en. is .hen streamed .o .he many subscnbers. RTCP enables 
,h. detection and transmission of only the read blocks affected by the streaming errors. 

FIG -y depicts another ponion of the interactive information d.stribudon system 100 
of FIG 1. This porfion of the system 100 compnses the stream caching server 102 and the 
30 infrastructure system manager 140 at ,he local head end 138, a stream caching server 
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and an ,nfras,ru«ure system n,anage, 204 a, a remote head end 206, and a backbone 
streaming network 210. The stream caching server 202 and the infrastructure system 
manager 204 at the remote head end 206 operates in a similar manner ,o the respective 
stream caching server 102 and the infrastructure system manager 140 a, the local head end 

5 138 that were previously described. 

The local infrastructure system manager 140 receives a request for a particular 
content selecon and determines whether a user requested content selection is stoted ,n the 
storage medium 148. If the request content is no, in the storage medium 148, the local 
.nfraLcture system manager 140 tdentifies a remote stream cachmg server 202 that stotes 
,0 the requested program content and provides a (server) request to the remote system 
manager 204. For example, a local system manager 140 in San Francisco may request 
content from another remote remotely located server 202 in Boston. 

m response to this server request, the local system manager 140 coordinates the 
streaming remote stream caching server 202 streams the requested program content over 
,5 the backbone streaming network 2,0 to the locai stream cachmg server 102. The content ,s 
,hen st,.amed to the subscriber. If the locai system manager 140 dete^ines that there a„ 
enough user requests above some predetermined threshold number, then the content from 
the ..mote stream caching server 202 is also stored in the local stream cach.ng server 102. 
FIG 4 depicts a flow diagram of a method 400 for implementing the preprocessmg 
,0 of content e.g., a video program, .n accordance ,0 one embodiment o, the present inventton. 
The method 400 assumes that a user or subscriber has aiteady downloaded an applet 154 
from a http server 115. Specifically, the method 400 stans at step 402 and proceeds to srep 
404 where preprocessing is ,nit,a,ed when the applet .54 is executed by the processor 150 
,n the computer teri^inal 1 16. At step 406, a query determines whether a user has 
25 purchased shelf space. Namely, step 406 determines whether the user has purchased 
storage space or use of a portion of the storage medium 146 at the stream caching server 
102 Step 406 may also cover situations where a user has access to shelf space. 

,f the user has not put^hased shelf space on the storage medium 148, the method 
400 proceeds to end at step 424. If the user has already purchased shelf space on the 
30 storage medium 148, the method 400 proceeds to step 408, where content is loaded from a 
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memory 152 of the computer terminal 1 16. The content may comprise any multimedia 
presentation, e.g., a home made move, created by the user. 

At step 410, the method creates metadata for the loaded content. The metadata 
contams mdexing information that enables the stream cach.ng server 102 to respond to user 
5 interactivity commands withm a group of pictures or approximately one half a second. 
Examples of user interactivity commands mclude fast forward (FF), rewind (REW), pause, 
stop bookmark and return to place. Additionally, the metadata includes mformat,on that 
enables the stream caching server 102 to determine the type of fUe and the resolution of the 
content. 

10 The method 400 proceeds to step 412 where a query determines whether to 

transcode content. If no transcoding is required, the method 400 proceeds to step 416. 

If transcodmg .s required, the method 400 proceeds to step 414 where the content .s 
transcoded into a format that .s supported by a viewer used to v.ew downstream content. 
Step 414 may convert both the format and bit rate of the program content. In one 

15 embodiment, the content may be transcoded (at a elementary stream level) from MPEG-1, 
MP3 AVI QuickTime or Movmg formed into MPEG-2 formatted packets. At step 416, 
the method 400 encapsulates the transcoded content, e.g., MPEG-2 format, into an IP 
packet format (at the transport stream level). As previously descnbed wUh respect to FIGS. 
2A and 2B, storage of content in this IP packet format minimizes the retrieval t,me when 

20 the stored content is retrieved from the storage medmm 146 and stream to the distnbut.on 

network 104. . 
The method 400 proceeds to step 418, where the transcoded content and metadata ,s 

uploaded from the computer term.nal 116 and mto the storage medium 148 of the stream 
cachmg server 102. Onpe the transcoded content is stored at the stream cachmg server 102, 
25 the method 400 enables the user (sender of the content) to establish or set perm.ss,ons at 
step 420 This step establishes a subset of users who may access the content from the 
stream caching server 102. The method 400 proceeds to step 422, where a HTML page ,s 
established at the http server 1 15 for access by other users. The method 400 ends a step 
424. 
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FIGs 5A and 5B depic, a flow diagram of a me.hod for accessing program content 
that was preprocessed and stored at the stream cach.ng server ,02. In one embodiment of 
the method 500. a user may access the full verston of the program content only after 
correctly entering the password and paying to view the content. 
5 The method 500 starts at step 502 and proceeds to step 504, where a user access a 

htm, page and selects a panicular program content to be played from the stream cach.ng 
server At step 506, the method 500 queries whether the user has entered the correct 
password as previousiy estabhshed by the owner of the program content. Step 506 may 
a,temativc,y query for a particular subscriber name or keyword. 
0 If the correct password is not entered, the method 500 ends at step 540. If the 

eot^ct password ,s entered, the method 500 proceeds to step 508. where a query 
determines whether .he user has a viewer or player to view the selected program content. 
Namely step 508 determines whether a correct type of viewer or player is detected at the 
subscriber term.na,. e.g., a computer termina, 1 ,6, ,22 or ,32. ,f the correct player is not 
15 detected, the method 500 proceeds to download the player at 510 and then proceeds to step 
517 If the con-ec. player ts detected, the method 500 proceeds d.rectly to step 5,2. 

At step 5,7 a query determines whether the down,oaded player supports playback 
(Of content) at full quality. If the player dc«s not support play.ng of content at full quahty, 
the method 500 proceeds to step 526. If the player supports playing of content at full 
20 quality, the method 400 proceeds to step 5 14, whete a que,, determines whether the user 
has paid to v,ew a full qualt.y version of the content, e.g., a program content file If the 
user has pa,d to view the full quality version of the program content, the method 500 
proceeds to retrieve the full IP formatted content from the stream caching server 102 at step 
516 and streams the retrieved content over the distribution network 104 at step 518. The 
.5 method 500 proceeds to step 520, where the data link converter „2. „8 or 126 extracts the 
content (at the transport level) from the IP packets, content or MPEG-2 packets from the IP 
formatted content. At step 522. a query determines whether to transcode the extracted 
content Such transcoding is required to satisfy constraints in (downstream) transmitting 
the content over the access network .06, 108 and 1 10, and for playing the content on the 
30 viewer. If transcoding is not required, the method 400 proceeds to step 536. If transcoding 
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is required, the method 400 transcodes the content at normal quality at step 524 and 

proceeds to Step 536. 

Refemng back at step 5 14, if the user has not fully paid to view the full quahty 
version of the content, the method 500 proceeds to step 526. At th,s step 526, the method 

5 500 proceeds to retneve a sample or predefined portion of the IP formatted content (f.le) 
from the stream caching server 102 at step 526. The method 500 then proceeds to stream 
the retrieved portion over the d,stnbut.on network 104 at step 528 and extract the content, 
e g MPEG-2 packets, from the IP formatted content at step 530 . The method 500 
proceeds to step 532, where a query determines whether to transcode the extract content for 

10 transmission, over the access network 106, 108 or 1 10 and for playback on the viewer or 
player If no transcodmg .s required, the method 500 proceeds to step 536. If transcodmg 
is required, the method 550 proceeds to transcode the partial content at low quality at step 

534 and then to step 536. 

At step 536 the method 500 sends the transcoded content the subscriber termmal, 
15 e g a computer terminal 1 16, 122 and 132 v,a the access network 106, 108 and 1 10. The 
method 500 proceeds to play eUher the full or part.al quaUty content at step 538 and ends at 

Step 540. rut 
FIG 3 depicts a data structure useful in understanding an embodiment of the present 

invention. Specifically, FIG. 3 depicts a content stream 310 including a plurality of index 

20 points denoted as I„ I,, and so on up to I. (collectively index points I). It will be 

appreciated that while the index points I. through I. are depicted as being spaced in an 

approximately even manner, there is absolutely no requirement for such equal spacing of 

the indices. Each index point represents an appropnate stream entry point beginning with, 

for example, an I-frame in the case of an MPEG content stream. Each indexed portion may 

25 be long or short, may comprise for example an entire scene or a plurality of scenes, other 

index divisions may be used. 

FIG 3 also depicts a sub-stream 320 comprising a plurality of index points denoted 
as 1. i, and so on up to „. For puiposes of this discussion, it is assumed that the sub-stream 
comprises portions of content between index point I, and index point I„ yet not portions of 
30 the content traversing those index boundaries. The shaded portion of content stream 3 10 
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depicts the content within the sub-stream. The shaded portions of the sub-stream 320 
depict content that is not included within the sub-stream. It is noted that the sub-stream 
may be stored as a separate entity along with the content stream 310. The sub-stream may 
comprise those image frames associated with a portion of content indexed according to the 
5 MPEG-7 standards. For example, in the case of a movie having a plurality of scenes, 

where objects within the scenes of the movie have been indexed according to the MPEG-7 
format, a particular object (e.g., a sunset, classic automobile or actor) may be associated 
with an object or a group of objects represented within the content stream 310. Thus, the 
sub-stream 320 stores content specifically associated with a desired object so that such 

10 content may be retrieved. 

It is noted that in retrieving a sub-stream it is desirable for the first frame, for 
example an MPEG-2 stream, to comprise an intra-coded frame (an I-frame). As such, since 
it is possible for the desired content to be included in the content stream 3 10 at a point 
comprising a non I-frame, the sub-stream 320, when created, includes transcoding of at 
15 least the first frame within the sub-stream into an intra-frame coding format. Additionally, 
if desired, each of the index points within the sub-stream may be reencoded to insure that 
these points also comprise I-frames. It is noted that many sub-streams may be generated 
and stored and associated with the main content stream. 

In one embodiment of the invention, local content is loaded onto a client, such as a 
.0 set-top terminal or computer, from a local content source such as a video cassette recorder 
(VCR), digital video recorder (DVR), personal video recorder (PVR), computer or other 
storage and/or content source. For example, local content may be loaded onto the set top 
terminal or streamed to the set top terminal from a camera, live video and/or live audio 
feed. The content loaded onto the set top terminal is transcoded using a transcoding 
25 application. If the client does not have an appropriate transcoding application, such 

transcoding application is downloaded from a server within the system. The transcoding 
application is used to transcode the loaded content into a desired player format. For 
example, if the loaded content comprises streaming video and audio at baseband, then such 
streaming video and audio may be encoded according to any one of the formats previously 
30 discussed (e.g., AVI, MPEG, etc.). In the case of the locally loaded content being encoded 
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according to a first format that is not desired, then the content so encoded is transcoded 
using the transcoding appHcation to produce content in the desired player format. The 
transcoded local content is then encapsulated in a desired transport format. The desired 
transport format is a transport format adapted to a particular access network. The particular 

5 formats associated with various access networks are described above. The transport 

encapsulated content is then encapsulated in a realtime protocol packet, such as described 
above. The RTP packet stream is then uploaded to the server for subsequent distribution to 
clients (i.e., set-top terminals or computers) utilizing the desired player format via access 
networks utilizing the desired access network transport format. 

10 In addition to uploading encapsulated data to the server, the client may also provide 

access right data to the server for subsequent use in determining who may view the content, 
who may use the content, how long the content may be viewed, how long the content may 
be used, which geographic region the viewer or user is in, which set or sub-set of clients 
within a particular system are to have access, which passwords, if any, are to be used and so 

15 on. 

As an example, in the case of a set-top terminal associated with a user wishing to 
share video imagery of a new baby, the video imagery and associated audio information 
may be input to a set-top terminal from the video camera. The set-top terminal then uses a 
coding or transcoding application to encode the video and audio information according to a 

20 desired format, such as the above-mentioned AVI fomnat. Assuming that the subscribers 
within a system who are to receive this imagery (e.g., friends and neighbors) are within a 
system having an access network utilizing the MPEG-2 transport format, the set-top 
terminal or client will then transport encode the AVI encoded content to produce an 
MPEG-2 transport stream. The MPEG-2 transport stream comprising the AVI encoded 

25 content will be further transport encoded according to the realtime protocol techniques 
described above. The RTP encoded content and associated access information (e.g., 
passwords for family members and the like) is then uploaded to the server for subsequent 
distribution on demand to the appropriate viewers or users. 
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Although various embodiments which incoiporate the teachings of the present 
invention have been shown and described in detail herein, those skilled in the art can 
readily devise many other varied embodiments that still incorporate these teachings. 



